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(57) Abstract 



Process for the measurement of three dimensional (3D) profiles by means of the projection of structured light, based on the measurement 
of the shift which each direction of projection undergoes due to the height of the object under measurement The process includes the 
steps of determining the shift in pixel of the light codings, and/or of making sinusoidal the projected fringes, and/or of adding an automatic 
calibration procedure and/or of adding an automatic procedure which determines the geometrical and optical set-up of the system, given 
as input the required measurement constraints, and/or of integrating the measurement method called GCM with the measurement method 
called SGM. 
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Title 

Process for the measurement of three dimensional (3D) profiles by means 
of structured light projection. 

Description 

This invention relates to a process for the measurement of three 
dimensional (3D) profiles of objects of variable shape by means of the 
projection of structured light. 



As it is well known, the procedure for the measurement of 3D profiles is 
based on active stereo-vision systems, aimed at measuring the 3D 
profiles of target objects, and at representing them by means of a point 
cloud in which each single point represents the X- Y-, and Z- coordinates 
of the target object at that point. 

It is well known that such a 3D measuring apparatus comprises three 
basic components, shown in the layout of Figure 1 : 

the acquisition unit, represented for example by a black and white video- 
camera, possibly equipped with suitable zoom optics; 

the projection unit, represented for example by a projector based on a 
Liquid Crystal Display (LCD); 

the elaboration unit, represented by a Personal Computer for example of 
the Pentium family, equipped with a suitable frame grabber, on which the 
measurement procedures are stored. 

Figure 1 also shows the optical geometry of such a system. A black and 
white CCD video camera with 756x581 picture elements acquires the 
patterns projected by the projection unit, along a different direction with 
respect to that of projection. Points Ec and Ep, which represent the 
entrance and exit pupils of the video camera and of the projector 
respectively, are at a distance L from reference plane R and at a distance 
d from each other. The optical axes of the video camera and of the 
projector define a plane which perpendicularly intersects plane R, and 
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intersect each other at point O on plane R. Parameters FW and FH 
represent the width of the field of view along the x-coordinate and the y- 
coordinate respectively. A Personal Computer controls the projector and, 
through a suitable frame grabber, acquires the images and performs the 
elaboration. The image resolution is N pixels along the x- coordinate, and 
M pixels along the y- coordinate. The gray level range is 256. 

The 3D profile is obtained by means of trianguiation, by observing that 
the direction of projection described by ray EpA is seen by the video 
camera at point A on plane R when the object is absent, and at point B 
when the object is present. The height ZH(x,y) = HK of the object at point 
H(x,y) can be evaluated by observing that triangles BHA and EpHEc are 
similar, and that the following trianguiation equation holds; 

Zw(x ' y) -d + S R (x,y) < 1 > 

In Eq.(l) , SR(x t y) equals seg ment 

AB, and represents the shift that the direction of projection EpA 
undergoes on plane R due to the object height in point H(x,y). 

The sensor integrates four measurement methods which differ to each 
other in the approach used to evaluate shift SR(x,y). These are: 

1. measurement method based on gray code projection (GCM: Gray 
Code Method); 

2. measurement method based on the projection of a single grating 
(SGM: Single Grating Method); 

3. measurement method based on phase shift (PSM: Phase Shift 
Method); 

4. combined method (GCM-PSM: Gray Code Method-Phase Shift 
Method). 

GCM is based on the projection, at n successive steps, of n different 
fringe patterns of rectangular profile, to define in the work space 2 n 
directions of projection, called light planes. The stripes are parallel to the 
Y- coordinate. The spatial period of the stripes along the X- direction 



WO 99/28704 



PCT/EP98/07705 



3 



varies in such a way that a univocal correspondence is established 
between the light planes and the words of an n-bit Gray code. Figure 2 
schematically shows the resulting attribution of the coding; moreover, it is 
highlighted that the relation established by the n-bit code words allows the 
determination of 2 n different directions of projection. 

Each pattern is acquired by the video camera and suitable thresholding 
associates either the logic value 'O 1 or T to the gray level of each CCD 
pixel. This procedure results into a matrix, called Bit Plane Stack (BPS), 
having the same dimensions as the CCD: each element on BPS stores 
the n-bit Gray code word seen by the corresponding pixel on the CCD. A 
simple Look-up Table operation allows the conversion of this quantity to 

the integer light plane number /(/,*)> where j and k are the discrete 
coordinates on BPS, and correspond respectively to the X- and Y- 
coordinates on the reference plane as follows: 



The measurement procedure consists of the projection of the same 
pattern sequence in two different situations, corresponding to the case of 
absence and presence of the target object on plane R. Two matrices are 
formed, called BPS_R and BPS_0 respectively. Light planes undergo a 
displacement between BPS_R and BPS_0. Figure 3 gives an idea of (i) 
how light plane number !(j,k)=0101 is formed on BPS_R due to the 
projection on plane R of four patterns (GC_0, GC_1, GC_2, and GC_3, in 
the figure), and (ii) how it is shifted in BPS_0 when an object of 
parallepipedic shape is supposed to be the target. Shift Sp(x,y) is 
evaluated by means of the following formula: 




j=1 N 



y = (k-1/2)^ 
(3) 



k=1 



S R (x,y) = D c ak)^ 
(4) 
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where parameter D c G,k) represents the distance between codings and 
is obtained by calculating the point by point difference of the light plane 
codings on BPS_R and BPS_0: 

D c (j, k) = K- [BPS_ 0(j, k) - BPS_R(j, k)] 
(5) 

where K is a conversion factor from light planes to pixels. Shift S R (x,y) 
is then converted into the height by using Eq. (1). 

This technique was demonstrated to be particularly appropriate for 
dealing with steep slope changes and in general shape discontinuities of 
the surface to be measured, as the evaluation of the height is performed 
punctually, thus eliminating any influence on the measurement from 
neighborhood conditions. Moreover, the binarization of the measurement 
area results into high repeatability of the measurement, and leads to 
higher values of the measurement precision than those obtained by PSM 
and SGM. 

The methods based on the projection of a single grating (SGM) and on 
Phase Shift (PSM) associate phase values to light directions, as 
schematically shown in Figure 4, where it is also highlighted that the same 
phase value, denoted by 9, is associated to different light directions, 
making the correspondence non univocal. 

Both these method allows the achievement of two phase maps, for the 
reference and the object respectively, and differ only in the projection 
sequence and the technique used to perform the phase demodulation. 

PSM follows the well-known approach based on the projection of n p 
periodic patterns with spatial period p, where each pattern is obtained by 
spatially shifting the preceding one of a fraction p/n p of the period. The 
projected fringes must present a sinusoidal intensity profile. The 
measurement takes place in two steps: the intensity patterns are firstly 
projected onto plane R and then the same projection sequence takes 
place at the target. 

Denoting with lR e f,i(j,k) and lobj.iG.k) the intensity fields acquired by the 
video camera in the absence and in the presence of the object 
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respectively, in correspondence with each shift (i=0, 1 , 2, 3), they can be 
expressed by the two following relationships: 

RegO t k) = a r (j t k)+^^cos((o x (j + r(j))-i|) 
(6) 

o W (Lk) = a 0 (j f k)+^^ 

(7) 

1=0,1,2,3 



In Eqs.(6) and (7), a r (j,k) and a 0 (j t k) are the average brightness, b r Q\k) 
and b 0 Q,k) take into account for the fringe contrast for the two cases, o> x is 
the spatial frequency of the sinusoidal patterns along the x- direction and 
r(j) is the lateral shift of the patterns due to the crossed optical axes 
geometry. Parameter sQM) in Eq. (7) is the displacement due to the object. 
The acquired intensity fields can be combined to get the reference map 
0 Ref (j,k) and the object map <I>obj(i.k). These phase maps are evaluated 
by means of the two following relationships: 



^Ref(j.k) = <D x [j + r(j)] = atan 
(8) 



-1 ^Ref.1 ~ ^Ref.3 
'Ref,0 ~'Ref,2 



*obi(j.k) = a> x [j + r(j) + s(j,k)] = atarT 1 J 0 * 1 '° b< - 3 

'Obj,0 { Ob\,2 

(9) 

Shift S R (x,y) can be evaluated by means of: 

S R (x, y ) = l^tt k) - <D Re r a k)] • ™ = D c (j, k) ■ ^ 
(11) x 

where term — [OobjC.kJ-^RefG.k)] represents distance D c (j,k) between 
the phase codings. 

In SGM, a single grating of rectangular profile and period p is projected 
both on the reference and on the object to be measured; as schematically 
shown in Figure 5, it is then demodulated in the time domain by 
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multiplying each row of the acquired pattern by a sine and a cosine 
function of the same period; the resulting two signals are then lowpass- 
filtered and divided by each other. Finally, by evaluating the inverse 
tangent of this ratio, two phase maps are obtained, respectively called 
<D Ref (j f k) for the reference and <I>objG.k) for the object, which store, for 
each element (j f k), the phase value describing the light direction seen by 
the video camera at the pixel of position Q,k). Shift S R (x,y) is evaluated by 
means of Eq. (11). 

It is well known that the use of phase values to express directions of 
projection introduces an ambiguity in the determination of the 3D range 
information, which is strictly related to the fact that the phase is evaluated 
mod 2tc. Thus, robust unwrapping is needed to evaluate the absolute 
value of the phase. On the other hand, the phase is continuously 
distributed on the recorded scene: a good height resolution can be 
achieved, practically limited only by the errors due to the gray level 
quantization and to noise. 

The experimental work performed shows that good results can be 
obtained, provided that the object profile is smooth: in this case, the 
frequency content of the object is in the lower frequency range, and well 
separated from the grating carrier frequency. However, whenever the 
maximum phase slope is greater than 2tc/3p, being p the grating pitch, the 
unwrapping of the mod 2k fringe pattern fails, and the absolute phase 
value cannot be evaluated. 

The combination of GCM and PSM is aimed at adding a fractional term 
to the integer number /(/,*) coming from GCM, in order to further 
decompose each direction of projection. This is obtained by omitting the 
last pattern from the GCM sequence, by computing the least significant bit 
(LSB) of the resulting, incomplete coding by means of PSM and by adding 
a fractional part to the result. As an example, the projection sequence for 
the case n=4 is shown in Figure 6. The projection sequence is the same 
as that used in GCM, where the last pattern (GC_3) is omitted, because it 
is equal to the first pattern of the PSM sequence. The projection of the 
complete PSM sequence allows the achievement of the phase value. As a 
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result, real valued light plane number is determined: in the following it is 
denoted by l(x,y), and expressed by: 

Kj,k) = iG.k) + ^[<D(j,k)+ A<*(j,k)] (12) 

IE 

In Eq. (12), term l{j y k) represents the incomplete coding, obtained 
from GCM, with the omission of the last pattern. The phase term <I>G\k) is 
evaluated by means of PSM, without the application of the phase 
unwrapping algorithm. A<I>(j,k) represents the phase correction term 
necessary to achieve the complete encoding, and is evaluated 
experimentally. 

This method allows to further decompose the coding of the light 
direction from the projector and to code them univocally along the field of 
view, as schematically shown in Figure 7. The measurement takes place 
in two steps: the pattern sequence is projected firstly on the reference and 
then on the object. Two matrices of real values are obtained, called 
RVLP_R arid RVLP_0 for the reference and for the object respectively. 
Shift S R (x,y) is then calculated by means of the following relation: 

S R (x,y) = K.[RVLP.O(j,k)-RVLP^R(j,k)]^ = D c ak)^ (13 ) 

where term K-[RVLP_0(j,k)-RVLP_R(j,k)] is coding distance D c (j,k). 
The height information is then obtained by means of relation (1). 

This procedure allows the measurement of objects presenting 
discontinuities of shapes as well as small details of the surface. In fact, 
shift S R (x,y) is obtained by using expression (13), where term Dc(j,k) is a 
real number instead of an integer, and results into an increased resolution 
with respect to GCM. 

Limitations. 

The presented measurement procedure, when performed by using the 
methods above described, shows a number of drawbacks, in dependence 
of each single approach. 

Limitations of the GCM approach. 

Limitation on the width of the measurement area due to the resolution. 
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The triangulation formula (1) shows that the height resolution depends, 
among the other parameters, on the minimum value of shift S R (x,y). In 
turn, the minimization of S R (x,y) depends on the reduction of quantities 
D c (j f k) and FW in Eq. (4). Dc(j f k) is an integer number, and can not be 
further reduced. Consequently, the minimization of shift S R (x,y) can be 
obtained by reducing value FW, being constant the video camera 
resolution (N). 

Limitation on the width of the measurement area due to the slope error. 

The dimension of the measurement area is further reduced by the 
presence of an error in the measured height which depends on the so 
called crossed optical axes geometry of the system, where the projection 
occurs at finite distance from the reference (see Figure 1). Figure 12 
shows an example of the measurement of an object of parallepipedic 
shape (Figure 12.a) and nominal dimensions 137 x 50.4 x 50 (mm). The 
nominal height is given with uncertainty of 100 jam. Figure 12.b plots the 
object profile measured along a single section: an error is well observable 
in the measured height, which presents a slope in correspondence with 
the top, flat surface of the target. This error is referred to as 'slope error 1 ; It 
is a systematic error, and directly depends on the fact that in Eq. (4) shift 
S R (x,y) is evaluated as a linear function of coding distance D c (j,k). 
However, this linear mapping can be successfully applied only if the fringe 
spatial period p R can be considered constant on reference R: in fact, in 
this case, S R (x,y) linearly depends on difference D c (j,k) and on period p R . 
However, this hypothesis is true only if points Ep and Ec are at infinity; it 
still holds as far as distance L is sufficiently large and distance d 
sufficiently small to limit the dependence of p R on X-. In contrast, for the 
geometrical setups characterized by reduced values for L and increased 
values for d, as those typical of GCM, any attempt to evaluate S R (x,y) by 
using constant p R inherently yields an error in the estimation of the height. 
This error is particular evident when flat surfaces are measured; in fact, 
the measured profile reveals a slope increasing with both X- and Z-. 

Limitation of the measurement area due to the precision of the 
measurement. 
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The shift of the codings on matrix BPS_0 with respect to their position 
on BPS_R not only depends on the deformation induced by the object 
shape, it also depends on the level of uncertainty introduced by the 
binarization of the gray levels. Since lateral resolution N is a finite value, a 
number of consecutive elements in the two matrices store the same 
coding. In the following K^p will be used to indicate this number , with 
reference to direction of projection LP. The experimental work carried out 
so far highlighted the dependence of K LP on the threshold value used 
during the binarization process and the fact that Kip can be different in 
BPS_0 with respect to BPS_R, even for same coding LP. Due to the fact 
that the comparison between the two matrices is performed by means of a 
simple difference, it is impossible to isolate and to compensate the 
contribution on the height errors due to the uncertainty of parameter K LP . 
This fact results in an increased inaccuracy of the measurement. This 
effect is well observable in the saw-tooth behavior of the profile shown in 
Figure 12.b. It increases with Kl P , and, for constant N, with increasing 
values of FW. 

Limitations of the approach based on PSM. 

Need of a customized projector 

The projection sequence of PSM is based on fringe patterns of 
sinusoidal profile, which can be produced by means of rather 
sophisticated and expensive projectors. 

Limitation on the width of the; measurement area due to the slope error. 

This limitation is the same as that described for GCM. This is an 
obvious consequence of the fact that the slope error is due to the crossed 
optical axes geometry of the system, which is the same for all the 
methods. Figure 13 shows an example of the slope error in the height 
evaluated by means of PSM. The same object as in Figure 12.a is 
measured: obviously the actual height is not revealed by PSM, due to the 
ambiguity of the phase. 

Limitations of the approach SGM. 

Limitation on the width of the measurement area due to the slope error. 
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This limitation is the same as that described for GCM and for PSM. 
This is an obvious consequence of the fact that the slope error is due to 
the crossed optical axes geometry of the system, which is the same for all 
the methods. 

Limitations of the combined approach GCM-PSM. 

Need of a customized projector. 

The projection sequence of GCM-PSM is based on fringe patterns of 
both rectangular and sinusoidal profile, which can be produced by means 
of rather sophisticated and expensive projectors. 

Limitation on the width of the measurement area due to the slope error. 

This limitation is the same as that described for GCM, PSM and SGM. 
This is an obvious consequence of the fact that the slope error is due to 
the crossed optical axes geometry of the system, which is the same for all 
the methods. Figure 14 shows an example of the slope error in the height 
evaluated by means of GCM-PSM. The same object as in Figure 12. a is 
measured: the influence of the slope error is the same as that one 
observed in Figure 12.b, for GCM. 

The object of the invention is to develop a process for the 
measurement of three dimensional profiles of various objects, such that, 
limitations and drawbacks of the known methods above described can be 
overcome, and in particular the limitation related to the width of the 
measurement area due to accuracy and/or precision. 

In addition, the object of the invention is to provide a procedure for the 
measurement of three-dimensional profiles which uses the same device of 
projection for all the measurement methods and, in particular, avoids the 
use of sophisticated devices to project fringe patterns of sinusoidal profile. 

These and other objects, together with their related advantages 
described in the following documentation, are obtained by a process for 
the measurement of three dimensional (3D) profiles by means of the 
projection of structured light, based on the measurement of the shift 
(S R (x,y)) that direction of projection EpA undergoes due to the height 
(ZH(x.y)) of the object under measurement, exploiting the method based 
on the Gray Code projection (GCM), and/or the method based on the 
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projection of a single grating (SGM), and/or the method based on the 
Phase Shift (PSM), and/or the method based on the combination of GCM 
with PSM (GCM-PSM). This process, in accordance to the invention, 
includes the following steps: 

- the achievement of a reference matrix REF and of an object matrix OBJ 
and the determination of the shift that the codings of light directions 
undergo between matrix REF and matrix OBJ, and/or 

- the achievement of a sinusoidal profile of the projected fringes, by 
means of suitable lowpass filtering, and/or 

- the introduction of a procedure for the automatic calibration which, 
starting from a suitable model of the optical geometry of the system, 
allows the determination of the unknown parameters of such a model by 
means of an iterative process, and/or 

- the introduction of an automatic procedure for the determination of the 
geometrical and optical set up of the system, on the basis of the required 
measurement resolution and of the object shape and dimension, and for 
the determination of the method more suitable to measure the object 
among the four ones provided, and/or 

- the combination of the measurement method based on the projection of 
Gray Code (GCM) with the measurement method based on the projection 
of a single grating (SGM). 

More in detail, the process as in the invention, includes the following 
steps: 

- projection of the pattern sequence GCM and/or SGM and/or PSM and/or 
GCM-PSM on the reference; 

- achievement of reference matrix REF, where REF coincides with BPS_R 
in method GCM, with 0 Ref in method PSM and/or SGM, and with RVLP_R 
in method GCM-PSM; 

- projection of the pattern sequence GCM and/or SGM and/or PSM and/or 
GCM-PSM on the object; 
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- achievement of object matrix OBJ, where OBJ coincides with BPS_0 in 
method GCM, with <t> 0bj in method PSM and/or SGM, and with RVLP_0 in 
method GCM-PSM; 

- iterative comparison between the integer part of the elements in matrices 
REF and OBJ, and determination of the integer part S p j nt G,k) of shift 
SpQ.k) by means of the following formula: 

S pJrt ak) = r-j (10) 

where j* and j represent the indexes of the elements that store the 
same value of the integer part of the coding respectively in matrices OBJ 
and REF; 

- determination of fractional part Sp ( f rac G,k) of shift S p G,k).by means of the 
following formula: 



S Ptfrac (j,k) = 



OBJ(j,k)-REF(j,k + S Pttnt (j,k)) 



REF(j, k + (j, k)) + REF(j, k + S pJnt (j, k) - 1) (4Q) 

- determination of shift S p G,k) by means of the following formula: 

S p (j,k) = Sp iint ak) + S Pifrac (j t k) (41 ) 

- conversion of shift S p G,k) into shift S R (x,y) in millimeters by means of 
the following formula: 

S R (x,y) = S p (j,k).^ (14) 

- conversion of shift SR(x,y) into the height by means of Eq. (1 ). 

By means of this process the limitation on the width of the measurement 
area in methods GCM and/or PSM and/or SGM and/or GCM-PSM is 
removed: in fact, Eq. (14) represents a simple conversion from shift 
S p G,k), expressed in pixels and shift S R (x,y), expressed in millimeters; 
S p G,k) is obtained by means of an iterative search of each single coding, 
which takes into account for that coding LP is stored into K LP consecutive 
elements in each matrix, and that K|_p can be different in the two matrices. 
Figure 15 shows, for the object in Figure 12.a f the measurement obtained 
by using the described procedure (bold line) overlapped to that one 
obtained by using known methods (light line). It is worth noting that the 
procedure as in the invention results into a strong reduction of the saw- 
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tooth behavior in the measured profile: this is a direct consequence of the 
fact that the presence of the same coding into Kl P consecutive elements 
of the coding matrices is accounted for. 

Figure 16 plots the profile evaluated in correspondence with the same 
section as in the previous example when the described procedure is 
applied to method GCM-PSM (solid line). In the figure, this profile is 
compared to that obtained by using method GCM-PSM as in the state of 
the art. 

The validity of this procedure is observed even when PSM is used. 
Figure 17 shows an example. Figure 17.a depicts the target object. The 
profile measured by using PSM according to the state of the art is shown 
in Figure 17.b (dashed line), and compared to the profile evaluated when 
the procedure as in the invention is used in conjunction with PSM (solid 
line). The measurement errors in correspondence with the maximum 
height of the object in the two cases are respectively equal to 4.9% and 
0.16% of the nominal height. The efficacy of the correction increases with 
the measuring range along the z coordinate. This behavior is clearly 
shown in Figure 17.c, where the difference between the two profiles is 
plotted. 

In addition, the process as in the invention of achieving a sinusoidal 
profile of the projected fringes, includes the following steps: 

- projection of the pattern sequence PSM with fringes of rectangular 
profile on the reference; 

- evaluation of the mean spatial period of the fringes, denoted by T, by 
applying to a single row of one of the acquired intensity pattern the 
following formula: 



where max and min represent respectively the number of maxima and 
of minima detected on the row by using an algorithm for the evaluation of 
positive and negative peaks; 

- evaluation of the cut-off frequency of the lowpass filter by means of the 
following formula: 




(18) 
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* = 5T (17) 

- elaboration of terms AI Re fN Um and AI Re fDen b y means of the lowpass 
filter and evaluation of the ratio between the two resulting signals; 

- determination of reference matrix <I> Ref (j,k) by means of known Eq. (8); 

- projection of the pattern sequence PSM with fringes of rectangular 
profile on the object; 

- elaboration of terms AI 0 bj f Num and AlobjDen b Y means of the lowpass 
filter and evaluation of the ratio between the two resulting signals; 

- determination of object matrix <J>objG\k) by means of known Eq. (9); 

- evaluation of shift S R (x t y) by means of known relation (1 1) or (14); 

- evaluation of the height by means of known Eq. (1 ). 

By using this process a lowpass filtering operation is applied to the 
components at the numerator and at the denominator of each one of the 
following expressions: 



AfeelNun, _ U,-W 3 _ |>^ak) sin[n.co x (j + r(j))] 
Al«e«>e„ Wo - W.w f)B Re ^(j,k) • cos[n • a> x (j + r(j))] 



n=0 



AUnu. = Ui'Us B |>i.^ k )' $i ^^x(j^r( j )^s(j,k))] 
Alobipen W - lobi,2 £ Bobj „(j, k) • cos[n - co x G + r( j) + sa k))] 



(15) 



(16) 



n=0 



In Eqs. (15) and (16) terms l Re f,jG.k) and l 0 bj f iG.k) refer to the intensity 
fields acquired by the video camera in the absence and in the presence of 
the object respectively, in correspondence with each shift (i=0,1,2,3), and 
the presence of an infinite number of spectral components is evidenced, 
due to the rectangular intensity profile of the fringes. The difference 
operation which determines terms 

AlRewumi Afeef,Den, AI 0 bj (N um. Alobj.Den, eliminates the even spectral 

components while leaves all the odd components. Thus, the filtering 
operation is aimed at eliminating all the odd spectral components with the 
exception of that one at the fundamental frequency: this situation is the 
same as if the projected fringes had sinusoidal profile. 
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This procedure utilizes a Butterworth lowpass filter of the fourth order, 
implemented as an HR filter, using the well known bilinear transformation 
technique and compensating for the filter phase non linearity by filtering 
the signal forward and backward. The filter cut-off frequency is adaptive, 
since it is expressed as a function of mean spatial period T of the 
projected fringes, represented by Eq. (17). 

By means of this process the limitation on the need of a device 
specialized to the projection of sinusoidal fringes is removed; instead it is 
possible to use the device utilized to project the pattern sequences of 
GCM, SGM, which are all based on fringes of rectangular profile, even for 
the projection of both PSM and GCM-PSM pattern sequences. 

Figure 18.a shows the 3D profile reconstruction of an object obtained 
by using method GCM-PSM without using the filtering process. This 
profile should be compared with that one obtained by means of the novel 
filtering process, shown in Figure 18.b. The former profile (Figure 18.a) is 
characterized by a precision of 150 ^m, whilst the latter (Figure 18.b) 
shows a precision of 40 ^im. 

In addition, the process as in the invention, of introducing a procedure 
for the automatic calibration of the system, is based on the following 
steps: 

- projection of the pattern sequence GCM and/or SGM and/or PSM and/or 
GCM-PSM on the reference and on the object; 

- evaluation of light planes on the reference and on the object; 

- setting of the height tolerance Am; 

- evaluation of initial values oq of angle a, defined by the intersection 
between the projection plane within the projector and the reference plane, 
and of initial value y 0 of angle y, defined by the intersection between the 
first ray seen by the video camera and the perpendicular line to the 
reference plane, by using the following formulas: 




(22) 
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y 0 =atan 



J 



(23) 



- reconstruction of the profile of the calibration object by using estimates 
oto and y 0 ; 

- determination of those pixels which image the object; 

- evaluation of the linear regression of the calculated heights; 

- evaluation of a, +1 = (1 + p aj -m,)-a, with term p^O; 

- reconstruction of the profile of the calibration object by using current 
estimates of a and y; 

- evaluation of the linear regression of the calculated heights; 

- if the module of the resulting angular coefficient is greater than that one 
at the previous step, the sign of term p^ is inverted; 

- resetting of the initial conditions and execution of the same sequence of 

steps for the determination of angle y; if it is necessary, the sign of term 
p^ is inverted; 

- optimization cycle: 

while coefficient nij is greater than the preset tolerance Am, new values 
for angles a and y are derived by means of the following formulas: 
a^HI+Pa^m,).^ (24) 

Tw=(1 + P T |- m i)-Ti (25) 

- the linear regression on the measured profile is performed, angular 

coefficient m j+1 is evaluated, and the cycle convergence is checked; if the 

cycle diverges those values for angles a and y evaluated at previous 
iteration are assigned to current values, and parameters p aj and p Ti are 

decremented. 

This process allows the transformation of phase matrices <I>objG.k) and 
^RefO.k) in Eq.(11) and of real valued matrices RVLPJD(j,k) and 
RVLP_R(j,k) in Eq. (13) into abscissas matrices, according to the model of 
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the system optical geometry presented in Figure 8, by using the following 
expressions: 



X a (J,k) = L 



tan 



y -atan 



nlpsinjO^,) 



X R U,k) = L 



tan 



Y - atan 



nip-sin^,.,) J J) 

f REFU.kysinjSj , , ^ 
nlp-sinjoj C ° Sir a) 



(19) 



nip. sinie^) 



J J 



(20) 



where REF(j,k) and OBJ(j,k) are the codings of the light planes, and 
thus correspond to elements BPS_RG,k) and BPR_0(j,k) and/or to 
elements <D Ref (j,k) and OobjG.k) and/or to elements RVLP_R(j t k) and 
RVLP_OQ\k). Eqs.(19) and (20) are applied after the determination of the 
matrices storing the light plane codings. The whole elaboration scheme is 
shown in Figure 9. 

This process as in the invention increases the accuracy in the 
evaluation of the height, since shift S R (x,y) is determined by means of the 
following formula: 

[XottlO-XpO-k)] (21) 

which represents a simple difference between abscissa values. 

The use of Eqs. (19) and (20) is possible only if two new angles are 
known, called a and y. These are shown in Figure 8, and represent 
respectively the angle defined by the intersection between the projection 
plane within the projector and the reference plane, and the angle between 
the first light direction seen by the video camera (ray E p s in the figure) 
and the perpendicular line E p R to the reference plane. These angles are 
unknown and must be determined by means of a suitable calibration 
procedure. 

This procedure is iterative and automatically determines angles a and 
y. It is based on the minimization of a suitable parameter, correlated with 
the effect of the slope error on the measured height. This parameter is 
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denoted in the following by m, and represents the angular coefficient of 
the linear regression evaluated on the height measured on a calibration 
master of parallepipedic shape. Before the calibration of the profilometer, 
the system geometry is not precisely known, and the initial values for 
angles a and y are defined by considering the simplified geometry in 
Figure 10. The orientation of the projection plane is unknown: for 
simplicity it is initially assumed that it is perpendicular to the projector 
optical axis, with an uncertainty of ±15°. Thus, the initial estimate on angle 
a is determined by: 

(22) 



(d 0 

cc n = atan — 

Uo 



"0 J 

where L 0 and d 0 are initial estimates of parameters L and d. 

Initial value y 0 of angle y is evaluated by considering triangle EpSQ as: 

(23) 



n = atan 



d 0 - : 



L 0 



From this initial condition the calibration process evolves exploiting the 
information related to the angular coefficient m of the linear regression 
evaluated over the profile measured along a single section of the 
calibration master. Angles a and y are evaluated by means of Eqs. (24) 
and (25) in order to reduce step by step angular coefficient m. 

Values oq +1 and y j+1 are determined from current values oq and yj, by 
multiplying them respectively by factor + mj and factor + -m ( ) t 

where p^ and p^ are weights, experimentally determined. At each 

iteration, by using the new values for a and y the master profile is 
evaluated along the section, and the corresponding angular coefficient m 
is determined. This iteration stops when coefficient m is within tolerance 
Am. 

The significance of such formulas (24) and (25) is clear the higher the 
value of m, the higher the variation induced on values ctj +1 and y j+1 ; if, on 
the contrary, angular coefficient m is small, only small variations will be 
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produced from a\ and yj, to Oj +1 and y j+1l which are actually close to the 
search solution. 

By means of this procedure, the measurement area can be increased, 
since shift S R (x,y) is evaluated directly as the difference between 
abscissa values on the reference, as expressed by Eq. (21), instead of 
using difference D c (j,k) between codings of light planes. Figure 19 shows 
the height profile in correspondence with a single section of the object in 
Figure 12.a, measured by using such a calibration process in combination 
with GCM (solid line). In the figure, this profile is compared to that one 
resulting from the utilization of GCM without the proposed calibration 
(dashed line). The measurement error is reduced from 25.3% to 0.08% of 
the nominal height of the object. Figures 20 and Figure 21 show the effect 
of the introduction of such a calibration process in conjunction with 
method GCM-PSM (Figure 20) and method PSM (Figure 21). Both the 
figures plot the profiles measured by using the calibration process (solid 
line) and by conventional GCM-PSM and PSM (dashed line). In both the 
experiments a strong reduction of the measurement error can be 
observed, of the same entity as that evaluated in correspondence with 
GCM. 

In addition, the process as in the invention, of introducing an automatic 
procedure for the determination of the geometrical and optical set up of 
the system, and of the method more suitable to measure the object among 
the four ones provided, including the following steps: 

- input statement of the object dimensions denoted by X, Y, and Z; 

- input statement of the required measurement resolution z min ; 

- input statement of the shape characteristics of the object; 

- evaluation of the field of view FW by means of the following relation: 

FW=t7-max(X,Y) (26) 

- evaluation of parameter L by means of the following formula: 

L = 10Z (27) 

- evaluation of focal length of the video-camera by using the following 
formula: 
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f = k r L/FW (28) 
where k1 is experimentally determined; 

- evaluation of focal number 1# by means of: 

f#=k 2 -Zf 2 /L 2 (29) 
where k 2 is experimentally determined; 

- choice of the measurement method among GCM, PSM, SGM and 
GCM-PSM, in dependence on both the topology of the object and 
required z min ; 

- evaluation of the value of parameter d by using the following formulas: 

. FWL 

<3o) 

^ _ L » p - ^ffgn ~ P * ^min " ^min (31) 
2'P'Znun 

where p represents the spatial period of the projected fringe pattern 
presenting minimum period among those patterns of the chosen projection 
sequence, and <I> m j n is the minimum phase value achievable, depending 
on the acquisition device; Eq. (30) is used when GCM is selected to 
perform the measurement, while Eq. (31 ) is used in conjunction with the 
other methods; 

- output of the optical and geometrical parameters determined (f, f# t FW, 
L, d), and of the suggested projection method. 

Such a procedure makes the set-up phase of the system completely 
automatic, and considerably simplifies the use of the system, even for non 
skilled users. In fact, the evaluation of the critical parameters is performed 
by the process itself, which determines the best values for : 

- field of view FW; 

- distance of the projector from the video-camera (d); 

- stand-off distance L; 

- focal length f; 
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- the most suitable projection method among those available (GCM, 
PSM, SGM and GCM-PSM), 

in order to guarantee the required resolution. The structure of the 
process is schematically shown in Figure 1 1 . 

In addition, the process as in the invention, of introducing a procedure 
for the combination of the measurement method based on the projection 
of Gray Code (GCM) with the measurement method based on the 
projection of a single grating (SGM), is based on the following steps: 

- projection of (n-1) patterns of the pattern sequence of GCM on the 
reference; 

- evaluation of the incomplete coding of light planes k ef (j,k); 

- projection of the n-th pattern on the reference; 

- evaluation of phase OReffl.k) on the reference by using method SGM; 

- evaluation of matrix REF of real valued light planes on the reference. 
Denoting with lR e f(i> k ) the ©lenient stored into the matrix at position (j,k), it 
is evaluated by using the following formula: 



where A<£G,k) represents the correction term needed to eliminate the 
incompleteness of GCM; 

- projection of (n-1 ) patterns of the pattern sequence of GCM on the object; 

- evaluation of the incomplete coding of light planes b bj (j,k); 

- projection of the n-th pattern on the object; 

- evaluation of phase <&objti»k) on the object by using method SGM; 

- evaluation of matrix OBJ of real valued light planes on the object. 
Denoting with lobjC\k) the element stored into the matrix at position (j t k), it 
is evaluated by using the following formula: 



where A<D(j,k) represents the correction term needed to eliminate the 
incompleteness of GCM; 



l R ef(i.k) = kf(*k) + -[*R.c(j.k) + AO<j.k)] 

71 



(33) 



iobiak)=Ujak)+f[<i»obia.k)+AO(j,k)] 



(34) 
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- conversion of matrices REF and OBJ by means of Eq. (19) and (20); 

- evaluation of shift S R (x,y) by means of Eq.(21 ); 

- evaluation of the height by means of Eq.(1 ). 

Such a procedure allows the reconstruction of the 3D profiles of objects 
presenting steep slope changes as well as fine details as in method GCM- 
PSM, while reducing the number of the projection patterns from (n+np+1 ), 
as in GCM-PSM to value n. The procedure combines method GCM with 
method SGM, in such a way that two matrices REF and OBJ are obtained, 
for the reference and for the object respectively, storing real valued 
elements l(j,k) f expressed by: 

KlM) = Tak)+-[0(j.k) + A0>(j,k)] (35) 

In Eq. (35), term l(j,k) represents the contribution to the coding 
determined by GCM, whilst <X>(j,k) is the phase term evaluated by SGM. 

This procedure allows to further decompose the light directions from 
the projector with respect to the integer coding achievable by using GCM, 
and, at the same time, to univocally determine each direction of projection 
along the whole field of view FW. 

The procedure as in the invention of automatic calibration can be then 
exploited, to determine angles a and y, and then the light plane codings 
can be converted into abscissa values by using Eqs. (19) and (20). The 
use of Eq.(21) to determine shift S R (x,y) avoids the influence of the slope 
error on the measured height. 

The invention is illustrated with reference to the following drawings, 
given to exemplify and not to limit the invention itself, wherein: 

Figure 1 shows the commonly used layout of the system, together with 
its optical geometry; 

Figure 2 highlights the coding of light planes performed by means of 
GCM; 

Figure 3 shows an example of the GCM projection sequence of 4 
patterns on the reference plane (i) and in the presence of the measured 
object (ii); 
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Figure 4 shows the phase coding attribution to each direction of 
projection by using methods PSM or SGM: same phase value 6 is given to 
more than one light ray; 

Figure 5 schematically shows the SGM procedure 

Figure 6 shows an example of the projection sequence in the combined 
procedure GCM-PSM, for n equal to 4; 

Figure 7 highlights the coding of real valued light planes obtained by 
combining GCM to PSM; 

Figure 8 shows the advanced model of the optical geometry; 

Figure 9 data elaboration flow chart after the introduction of the novel 
projective model; 

Figure 10 geometric model used to determine the initial conditions on 
angles a and y; 

Figure 1 1 shows the flow chart of the optical CAD automatic procedure; 

Figure 12.a and 12.b respectively show a target chosen for the 
measurement and the profile along a single section measured by using 
conventional methods; 

Figure 13 shows the influence of the slope error when the method 
based on PSM is used; 

Figure 14 shows an example of the influence of the slope error when 
the method based on GCM-PSM is exploited; 

Figure 15 shows the correction of the slope error in the method based 
on GCM by using the search-based procedure; 

Figure 16 shows the correction of the slope error in the method based 
on GCM-PSM by using the search-based procedure; 

Figures 17.a, 17.b t 17.c show the correction of the slope error in the 
method based on PSM by using the search-based procedure; 

Figure 18 shows the effect of the filtering procedure used in PSM: 18.a: 
measurement performed without the filtering operation; 18.b: 
measurement performed with the filtering procedure; 
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Figure 19 shows the correction of the slope error in method GCM by 
means of the procedure for calibrating angles a and y; 

Figure 20 shows the correction of the slope error in method GCM-PSM 
by means of the procedure for calibrating angles a and y; 

Figure 21 shows the correction of the slope error in method PSM by 
means of the procedure for calibrating angles a and y; 
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Claims 

1. process for the measurement of three dimensional (3D) profiles by 
means of the projection of structured light, based on the measurement of 
the shift (S R (x,y)) that direction of projection EpA undergoes due to the 
height (Zj-ifoy)) of the object under measurement, exploiting the method 
based on the Gray Code projection (GCM), and/or the method based on 

v the projection of a single grating (SGM), and/or the method based on the 
Phase Shift (PSM), and/or the method based on the combination of GCM 
with PSM (GCM-PSM). This process, in accordance to the invention, 
includes the following steps: 

- the achievement of a reference matrix REF and of an object matrix OBJ 
and the determination of the shift that the codings of light directions 
undergo between matrix REF and matrix OBJ, and/or 

- the achievement of a sinusoidal profile of the projected fringes, by 
means of suitable lowpass filtering, and/or 

- the introduction of a procedure for the automatic calibration which, 
starting from a suitable model of the optical geometry of the system, 
allows the determination of the unknown parameters of such a model by 
means of an iterative process, and/or 

- the introduction of an automatic procedure for the determination of the 
geometrical and optical set up of the system, on the basis of the required 
measurement resolution and of the object shape and dimension, and for 
the determination of the method more suitable to measure the object 
among the four ones provided, and/or 

- the combination of the measurement method based on the projection 
of Gray Code (GCM) with the measurement method based on the 
projection of a single grating (SGM). 

2. A process, according to claim 1 , characterized in that it includes the 
following steps: 

- projection of the pattern sequence GCM and/or SGM and/or PSM and/or 
GCM-PSM on the reference; 
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- achievement of reference matrix REF, where REF coincides with BPS_R 
in method GCM, with 0 Ref in method PSM and/or SGM, and with RVLP_R 
in method GCM-PSM; 

- projection of the pattern sequence GCM and/or SGM and/or PSM and/or 
GCM-PSM on the object; 

- achievement of object matrix OBJ, where OBJ coincides with BPS_0 in 
method GCM, with <D 0b j in method PSM and/or SGM, and with RVLP_0 in 
method GCM-PSM; 

- iterative comparison between the integer part of the elements in matrices 
REF and OBJ, and determination of the integer part S p j nt G,k) of shift 
SpO.k) by means of the following formula: 

S pJfrt (j,k) = j*-j (10) 

where j* and j represent the indexes of the elements that store the 
same value of the integer part of the coding respectively in matrices OBJ 
and REF; 

- determination of fractional part S Ptfrac G,k) of shift S p Q,k) by means of the 
following formula: 

OBJa^-REFttk + S^gk)) 
P .ftacU. ) REF(j,k + S P(tnt ak)) + REF(j,k + S pJnt (j,k)-1) (4Q) 

- determination of shift S p (j,k) by means of the following formula: 

S p (j,k) = S Piint (j,k) + S Pifrac (j,k) (41 ) 

conversion of shift S p Q,k) into shift Sp(x,y) in millimeters by means of 
the following formula: 

S R (x,y) = S p (j,k).^ (14) 

- conversion of shift S R (x,y) into the height by means of Eq. (1 ). 

3. A process according to claim 1, characterized in that it achieves a 
sinusoidal profile of the projected fringes, including the following steps: 

- projection of the pattern sequence PSM with fringes of rectangular 
profile on the reference; 
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- evaluation of the mean spatial period of the fringes, denoted by T, by 
applying to a single row of one of the acquired intensity pattern the 
following formula: 



where max and min represent respectively the number of maxima and 
of minima detected on the row by using an algorithm for the evaluation of 
positive and negative peaks; 

- evaluation of the cut-off frequency of the lowpass filter by means of the 
following formula: 



- elaboration of terms AlR ef Num and AI Ref Den by means of the lowpass 
filter and evaluation of the ratio between the two resulting signals; 

- determination of reference matrix 4> Ref {j,k) by means of known Eq. (8); 

- projection of the pattern sequence PSM with fringes of rectangular 
profile on the object; 

- elaboration of terms AI 0 bj,Num and A 'obj,Den b y means of the lowpass 
filter and evaluation of the ratio between the two resulting signals; 

- determination of object matrix <&obj(J» k ) b y means of known Eq. (9); 

- evaluation of shift S R (x,y) by means of known relation (1 1 ) or (14); 

- evaluation of the height by means of known Eq. (1). 

4. A process according to claim 3, characterized in that it utilizes a 
Butterworth lowpass filter of the fourth order, implemented as an IIR filter, 
with adaptive cut-off frequency. 

5. A process according to claim 1 , characterized in that it introduces the 
automatic calibration of the system, based on the following steps: 

- projection of the pattern sequence GCM and/or SGM and/or PSM and/or 
GCM-PSM on the reference and on the object; 

- evaluation of light planes on the reference and on the object; 

- setting of the height tolerance Am; 




(18) 




(17) 
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- evaluation of initial values a 0 of angle a, defined by the intersection 
between the projection plane within the projector and the reference plane, 
and of initial value y 0 of angle y, defined by the intersection between the 
first ray seen by the video camera and the perpendicular line to the 
reference plane, by using the following formulas: 



a 0 =atan =t (22) 



y 0 = atan 



(23) 



- reconstruction of the profile of the calibration object by using estimates 
oq and y 0 ; 

- determination of those pixels which image the object; 

- evaluation of the linear regression of the calculated heights; 

- evaluation of a i+1 = (1 + p a . m^-a, with term p^O; 

- reconstruction of the profile of the calibration object by using current 
estimates of a and y; 

- evaluation of the linear regression of the calculated heights; 

- if the module of the resulting angular coefficient is greater than that one 
at the previous step, the sign of term p^ is inverted; 

- resetting of the initial conditions and execution of the same sequence of 

steps for the determination of angle y; if it is necessary, the sign of term 
p^ is inverted; 

- optimization cycle: 

while coefficient mj is greater than the preset tolerance Am, new values 
for angles a and y are derived by means of the following formulas: 

aM-O+Pi-nO-a, (24) 

Tw = (1+P T r m i)*Ti (25) 

- the linear regression on the measured profile is performed, angular 
coefficient m l+1 is evaluated, and the cycle convergence is checked; if the 
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cycle diverges those values for angles a and y evaluated at previous 
iteration are assigned to current values, and parameters p a| and p Ti are 

decremented. * 

6. A process according to claim 5, characterized in that phase matrices 
0>ObjG»k) and <D Ref (j,k) in Eq.(11) and of real valued matrices RVLP_OG,k) 
and RVLP_Rfrk) in Eq. (13) into abscissas matrices, according to the 
model of the system optical geometry presented in Figure 8, by using the 
following expressions: 

f r r r\r>v v l\_ ^(s: \ w\ 



tan 



Y - atan 



Y' ■ cos(k - a) 

nlpsm{e m ) 

'■, OB 'J (J, k) siniS '.) ~, x 

nlp-sin(e, M ) 



(19) 



tan 



y - atan 



cos! 



(r-cz) 



REF<J,k).sin(8 tot ) 

tot) 

1 + / 7 — V 2 ^ * sm{r - a) 



\\\ 



(20) 



where REF(j,k) and OBJ(j,k) are the codings of the light planes, and thus 
correspond to elements BPS_R(j,k) and BPR_0(j,k) and/or to elements 
^RefO.k) and O O bj0»k) and/or to elements RVLP_R(j,k) and RVLP_0(j,k). 
Eqs.(19) and (20) are applied after the determination of the matrices 
storing the light plane codings. 

7. A process according to claims 5 and 6, characterized in that shift 
Sp(x,y) is determined by means of the following formula: 

[X 0 (j,k)-X R (j,k)] (21) 

8. A process according to claim 1, characterized by an automatic 
procedure for the determination of the geometrical and optical set up of 
the system, and of the method more suitable to measure the object among 
the four ones provided, including the following steps: 

- input statement of the object dimensions denoted by X, Y, and Z; 

- input statement of the required measurement resolution z mln ; 

- input statement of the shape characteristics of the object; 
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- evaluation of the field of view FW by means of the following relation: 

FW=t7max(X,Y) (26) 

- evaluation of parameter L by means of the following formula: 

L = 10Z (27) 

- evaluation of focal length of the video-camera by using the following 
formula: 

f = k 1 -L/FW (28) 
where k1 is experimentally determined; 

- evaluation of focal number f# by means of: 

f#=k 2 -Z.f 2 /L 2 (29) 
where k 2 is experimentally determined; 

- choice of the measurement method among GCM, PSM, SGM and GCM- 
PSM, in dependence on both the topology of the object and required z min ; 

- evaluation of the value of parameter d by using the following formulas: 

d = £?Lk (30) 

d _ L-p-Q min --p-<P m i n -z^n 3 

where p represents the spatial period of the projected fringe pattern 
presenting minimum period among those patterns of the chosen projection 
sequence, and O min is the minimum phase value achievable, depending 
on the acquisition device; Eq. (30) is used when GCM is selected to 
perform the measurement, while Eq. (31) is used in conjunction with the 
other methods; 

- output of the optical and geometrical parameters determined (f, f#, FW, 
L, d), and of the suggested projection method. 

9. A process according to claim 1 , characterized in that it introduces a 
procedure for the combination of the measurement method based on the 
projection of Gray Code (GCM) with the measurement method based on 
the projection of a single grating (SGM), based on the following steps: 



WO 99/28704 



PCT/EP98/07705 - 



31 

- projection of (n-1) patterns of the pattern sequence of GCM on the 
reference; 

- evaluation of the incomplete coding of light planes kef(i.k); 

- projection of the n-th pattern on the reference; 

- evaluation of phase <D Ref (j,k) on the reference by using method SGM; 

- evaluation of matrix REF of real valued light planes on the reference. 
Denoting with iRefG.k) the element stored into the matrix at position (j,k), it 
is evaluated by using the following formula: 

lRef(j.k) = kfak) + -[0 Ref (j t k)+AO(j,k)] (33) 

where A<I>G f k) represents the correction term needed to eliminate the 
incompleteness of GCM; 

- projection of (n-1 ) patterns of the pattern sequence of GCM on the 
object; 

- evaluation of the incomplete coding of light planes bbj(i> k ); 

- projection of the n-th pattern on the object; 

- evaluation of phase <X> 0 bj(j» k ) on the object by using method SGM; 

- evaluation of matrix OBJ of real valued light planes on the object. 
Denoting with lobjG.k) the element stored into the matrix at position (j,k), it 
is evaluated by using the following formula: 

'obj(ik)=bbiak) + |[o 0bj ak) + AOak)] (34) 

where AO(j,k) represents the correction term needed to eliminate the 
incompleteness of GCM; 

- conversion of matrices REF and OBJ by means of Eq. (19) and (20); 

- evaluation of shift Sp(x t y) by means of Eq.(21 ); 

- evaluation of the height by means of Eq.(1 ). 
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